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How NP Got a New Definition: A Survey 
of Probabilistically Checkable Proofs* 
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Abstract 

We survey a collective achievement of a group of researchers: the PCP 
Theorems. They give new definitions of the class NP, and imply that com- 
puting approximate solutions to many NP-hard problems is itself NP-hard. 
Techniques developed to prove them have had many other consequences. 
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1. PCP theorems: an informal introduction 

Suppose a mathematician circulates a proof of an important result, say Rie- 
mann Hypothesis, fitting several thousand pages. To verify it would take you and 
your doubting colleagues several years. Can you do it faster? Yes, according to the 
PCP Theorems. He can rewrite his proof so you can verify it by probabilistically 
selecting (i.e., using a source of random bits) a constant number of bits — as low 
as 3 bits — to examine in it. Furthermore, this verification has the following prop- 
erties: (a) A correct proof will never fail to convince you (that is, no choice of the 
random bits will make you reject a correct proof) and (b) An incorrect proof will 
convince you with only negligible probability (2~^'^° if you examine 300 bits). In 
fact, a stronger assertion is true: if the Riemann hypothesis is false, then you are 
guaranteed to reject any string of letters placed before you with high probability 
after examining a constant number of bits, (c) This proof rewriting is completely 
mechanical — a computer could do it — and does not greatly increase its size. ( Caveat: 
Before journal editors rush to adopt this new proof verification, we should mention 
that it currently requires proofs written in a formal axiomatic system — such as 
Zermelo Fraenkel set theory — since computers do not understand English.) 
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ITR Grant. 
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This result has a strong ring of implausibiUty. A mathematical proof is invalid 
if it has even a single error somewhere. How can this error spread itself all over 
the rewritten proof, so as to be apparent after we have probabilistically examined 
a few bits in the proof? (Note that the simple idea of just making multiple copies 
of the erroneous line everywhere does not work: the unknown mathematician could 
hand you a proof in which this does not happen, yet that does not make the proof 
correct.) The methods used to achieve this level of redundancy are reminiscent of 
the theory of error-correcting codes, though they are novel and interesting in their 
own right, and their full imphcations are still being felt (see Section 1^ . 

1.1. New definition of NP 

The PCP Theorems provide interesting new definitions for the complexity 
class NP. (Clarification: the singular form "PCP Theorem" will refer to a single 
result NP = PCP(logn, 1) proved in 012], and the plural form "PCP Theorems" 
refers to a large body of results of a similar ilk, some predating the PCP Theorem.) 
Classically, NP is defined as the set of decision problems for which a "Yes" answer 
has a short certificate verifiable in polynomial time (i.e., if the instance size is n, 
then the certificate size and the verification time is n'^ for some fixed constant c). 
The following are two examples: 

3-SAT ~ satisfiable boolean formulae of the form AND of clauses of size at most 
3, e.g., {xi V -1X2 V xs) A {-^xi V a;2 V X3) A (^4). (The certificate for satisfiability is 
simply an assignment to the variables that makes the formula true.) 

MATH-THEOREMzFC = set of strings of the form (T, 1") where T is a mathe- 
matical statement that is a theorem in Zermelo Fraenkel set theory that has a proof 
n bits long. (The "certificate" for theoremhood is just the proof.) 

The famous conjecture P 7^ NP — now one of seven Millenium Prize problems 
in math — says that not every NP problem is solvable in polynomial time. In 
other words, though the certificate is easy to check, it is not always easy to find. 

The PCP Theorem gives a new definition of NP: it is the set of decision 
problems for which a "Yes" answer has a polynomial-size certificate which can be 
probabilistically checked using O(logn) random bits and by examing 0(1) (i.e., 
constant) number of bits in it. 

Our earlier claim about proof verification follows from the PCP Theorem, since 
MATH-THEOREMzFC is in NP, and hence there is a way to certify a YES answer 
(namely, theoremhood) that satisfies properties (a) and (b). (Property (c) follows 
from the constructive nature of the proof of the PCP Theorem in OE].) 

Motivated by the PCP Theorems, researchers have proved new analogous def- 
initions of other complexity classes such as PSPACE j22| and PH (43) . 

1.2. Optimization, approximation, and PCP theorems 

The P versus NP question is important because of NP-completeness (also, NP- 
hardness) . Optimization problems in a variety of disciplines are NP-hard 30. , and 
so if P 7^ NP they cannot be solved in polynomial time. The following is one such 
optimization problem. 
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MAX-3SAT: Given a 3-CNF boolean formula find an assignment to the variables 
that maximizes the number of satisfied clauses. 

Approximation algorithms represent a way to deal with NP-hardness. An al- 
gorithm achieves an approximation ratio a for a maximization problem if, for every 
instance, it produces a solution of value at least OPT /a, where OPT is the value 
of the optimal solution. (For a minimization problem, achieving a ratio a involves 
finding a solution of cost at most a OPT.) Note that the approximation ratio is 
> 1 by definition. For MAX-3SAT we now know a polynomial-time algorithm that 
achieves an approximation ratio 8/7 |4()j . 

Though approximation algorithms is a well-developed research area (see |38l 
I62|). for many problems no good approximation algorithms have been found. The 
PGP Theorems suggest a reason: for many NP-hard problems, including MAX- 
GLIQUE, CHROMATIG NUMBER, MAX-3SAT, and SET-COVER, achieving cer- 
tain reasonable approximation ratios is no easier than computing optimal solutions. 
In other words, approximation is NP-hard. For instance, achieving a ratio 8/7 — e 
for MAX-3SAT is NP-hard 

Why do the PGP Theorems lead to such results? Details appear in the sur- 
vey PP (and [Feige 2002], these proceedings), but we hint at the reason using 3SAT 
and MAX-3SAT as examples. Gook and Levin [531 0B] showed how to reduce any 
NP problem to 3SAT, by constructing, for any nondeterministic machine, a 3GNF 
formula whose satisfying assignments represent the transcripts of accepting compu- 
tations. Thus it is difficult to satisfy all clauses. Yet it is easy to find assignmenent 
satisfying 1 — o(l) fraction of the clauses! The reason is that a computation tran- 
script is a very non-robust object: changing even a bit affects its correctness. Thus 
the Gook-Levin reduction does not prove the inapproxiniability of MAX-3SAT. By 
providing a more robust representation of a computation, the PGP Theorems over- 
come this difficulty. We note that MAX-3SAT is a central problem in the study of 
inapproximability: once we have proved its inapproximability, other inapproxima- 
bility results easily follow (see [Q; the observation in a weaker form is originally 
from work on MAX-SNP [^). 

1.3. History and context 

PGPs evolved from interactive proofs, which were invented by Goldwasser, Mi- 
cali, and Rackoff (34j and Babai ,5| as a probabilistic extension of NP and proved 
useful in cryptography and complexity theory (see Goldreich's survey [31]), includ- 
ing some early versions of PGPs "5^. In 1990, Lund, Fortnow, Karloff and Nisan |2S! 
and Shamir showed IP=PSPAGE, thus giving a new probabilistic definition of 
PSPAGE in terms of interactive proofs. They introduced a revolutionary algebraic 
way of looking at boolean formulae. In restrospect, this algebraization can also 
be seen as a "robust" representation of computation. The inspiration to use poly- 
nomials came from works on program checking jl7j (see also |47l 1111 [T5] ). Babai, 
Fortnow, and Lund '7^ used similar methods to give a new probabilistic definition 
of NEXPTIME, the exponential analogue of NP. To extend this result to NP, 
Babai, Fortnow, Levin, and Szegedy [5j and Feige, Goldwasser, Lovasz, Safra, and 
Szegedy 26i studied variants of what we now call probabilistically checkable proof 
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systems (Babai et al. called their systems holographic proofs). 

Feige et al. also proved the first inapproximability result in the PCP area: if 
any polynomial-time algorithm can achieve a constant approximation ratio for the 
MAX-CLIQUE problem, then every NP problem is solvable in „o(iogiogn) y^^g^ 
This important result drew everybody's attention to the (as yet unnamed) area 
of probabilistically checkable proofs. A year later, Arora and Safra |3j formalized 
and named the class PCP and used it to give a new probabilistic definition of NP. 
(Babai et al. and Feige et al.'s results were precursors of this new definition.) They 
also showed that approximating MAX-CLIQUE is NP-hard. Soon, Arora, Lund, 
Motwani, Sudan, and Szegedy |2] proved the PCP Theorem (see below) and showed 
that MAX-SNP-hard problems do not have a PTAS if P ^ NP. Since the second 
paper relied heavily on the still-unpublished first paper, the the PCP theorem is 
jointly attributed to For surveys of these developments see p iSniH^ISU) . 

2. Definitions and results 

Now we define the class PCP. We will use "language membership" and "de- 
cision problem" interchangeably. A (r{n),q{rL))- restricted verifier ior a language L, 
where r, q are integer-valued functions, is a probabilistic turing machine M that, 
given an input of size n, checks membership certificates for the input in the follow- 
ing way. The certificate is an array of bits to which the verifier has random-access 
(that is, it can query individual bits of the certificate). 

• The verifier reads the input, and uses 0{r(n)) random bits to compute a 
sequence of 0{q{n)) addresses in the certificate. 

• The verifier queries the bits at those addresses, and depending upon what 
they were, outputs "accept" or "reject" . 

• 

yxe L 3 certificate H s.t. Pr[Mnaccepts] = 1, (2.1) 

Vx ^ L V certificate H, Pr[Mnaccepts] < 1/2 (2.2) 
(In both cases the probability is over the choice of the verifier's random string.) 

PCP(r(n), (7(n)) is the complexity class consisting of every language with an 
(r(n), (7(n))-restricted verifier. Since NP is the class of languages whose mem- 
bership certificates can be checked by a deterministic polynomial-time verifier, 
NP Uc>oPCP(0,n'=). The PCP Theorem gives an ahernative definition: NP = 
PCP(logn, 1). Other PCP-like classes have been defined by using variants of the 
definition above, and shown to equal NP (when the parameters are appropriately 
chosen). We mention some variants and the best results known for them; these are 
the "PCP Theorems" alluded to earlier. 

1. The probability 1 in condition (|2.HI may be allowed to be c < 1. Such a 
verifier is said to have imperfect completeness c. 

2. The probability 1/2 in condition l|2.2|l may be allowed to be s < c. Such a 
verifier is said to have soundness s. Using standard results on random walks 
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on expanders, it can be shown from the PCP theorem that every NP language 
has verifiers with perfect completeness that use 0{k) query bits for soundness 
2"'= (here k < O(logn)). 

3. The number of query bits, which was 0{q{n)) above, may be specified more 
precisely together with the leading constant. The constant is important for 
many inapproximability results. Building upon past results on PCPs and 
using fourier analysis, Hastad recently proved that for each e > 0, every 
NP language has a verifier with completeness 1 — e, soundness 1/2 and only 3 
query bits. He uses this to show the inapproximability of MAX-3SAT upto a 
factor 8/7 — e. 

4. The free bit parameter may be used instead of query bits This pa- 
rameter is defined as follows. Suppose the query bit parameter is q. After the 
verifier has picked its random string, and picked a sequence of q addresses, 
there are 2'^ possible sequences of bits that could be contained in those ad- 
dresses. If the verifier accepts for only t of those sequences, then we say that 
the free bit parameter is logt (note that this number need not be an integer). 
Samorodnitsky and Trevisan show how to reduce the soundness to 2~*'' 
using k free bits [SHI- 

5. Amortized free bits may be used ^^l- This parameter is lims_+o /s/log(l/s), 
where fs is the number of free bits needed by the verifier to make soundness 
< s. Hastad shows that for each e > 0, every NP language has a verifier 
that uses 0(log n) random bits and e amortized free bits. He uses this to show 
(using a reduction from ^ and modified by EH USD that MAX-CLIQUE is 
inapproximable upto a factor n^^^ . 

6. The certificate may contain not bits but letters from a larger alphabet E. 
The verifier's soundness may then depend upon E. In a p prover 1-round 
interactive proof system, the certificate consists of p arrays of letters from E. 
The verifier is only allowed to query 1 letter from each array. Since each letter 
of E is represented by [log |E|] bits, the number of bits queried may be viewed 
as [log |E|] . Constructions of such proof systems for NP appeared in [TI)1IIK1 
051 im 1771 ES] . Lund and Yannakakis _45j used these proof systems to prove 
inapproximability results for SETCOVER and many subgraph maximization 
problems. The best construction of such proof systems is due to Raz and 
Safra [S3- They show that for each k < y/logn, every NP language has a 
verifier that uses O(logn) random bits, has log|E| = 0{k) and soundness 
2^*^. The parameter p is 0(1). 

3. Proof of the PCP theorems 

A striking feature of the PCP Theorems is that each builds upon the previous 
ones. However, a few ideas recur. First, note that it suffices to design verifiers 
for 3SAT since 3SAT is NP-complete and a verifier for any other language can 
transform the input to a 3SAT instance as a first step. The verifier then expects 
a certificate for a "yes" answer to be an encoding of a satisfying assignment; we 
define this next. 



642 



S. Arora 



For an alphabet S let E"* denote the set of m-letter words. The distance 
between two words x,y £ S™, denoted 6{x,y), is the fraction of indices they differ 
on. For a set C C S™, let the minimum distance of C, denoted min-dist(C), refer to 
mhij: y(zc;x^y {S{x, y)} and let 6{x, C) stand for min^gc {H^^ u)}- If niin-dist(C) = 7, 
and (5(x,C) < 7/2, then triangle inequality implies there is a unique y G C such that 
S{x,y) — S{x,C). We will be interested in C such that min-dist(C) > 0.5; such sets 
are examples of error- correcting codes from information theory, where C is thought 
of as a map from strings of log \C\ bits ("messages") to C. When encoded this way, 
messages can be recovered even if transmitted over a noisy channel that corrupts 
up to l/4th of the letters. 

The probabilistically checkable certificate is required to contain the encoding of 
a satisfying assignment using some such C. When presented with such a string, the 
verifier needs to check, first, that the string is close to some codeword, and second, 
that the (unique) closest codeword is the encoding of a satisfying assignment. As 
one would expect, the set C is defined using mathematically interesting objects 
(polynomials, monotone functions, etc.) so the final technique may be seen as 
a "lifting" of the satisfiability question to some mathematical domain (such as 
algebra). The important new angle is "local checkability," namely, the ability to 
verify global properties by a few random spot-checks. (See below.) 

Another important technique introduced in |2] and used in all subsequent pa- 
pers is verifier composition, which composes two verifiers to give a new verifier 
some of whose parameters are lower than those in either verifier. Verifier com- 
position relies on the notion of a probabilistically checkable split- encoding, a no- 
tion to which Arora and Safra were led by results in [Hj. (Later PCP Theorems 
use other probabilistically checkable encodings: linear function codes 0, and long 
codes [I'M Kffil r?7 One final but crucial ingredient in recent PCP Theorems is 
Raz's parallel repetition theorem j53j . 

3.1. Local tests for global properties 

The key idea in the PCP Theorems is to design probabilistic local checks that 
verify global properties of a provided certificate. Designing such local tests involves 
proving a statement of the following type: if a certain object satisfies some local 
property "often" (say, in 90% of the local neighborhoods) then it satisfies a global 
property. Such statements are reminiscent of theorems in more classical areas of 
math, e.g., those estabhshing properties of solutions to PDEs, but the analogy is 
not exact because we only require the local property to hold in most neighborhoods, 
and not all. 

We illustrate with some examples. (A research area called Property Testing 
now consists of inventing such local tests for different properties.) There is a set 
C C S™ of interest, with min-dist(C) > 0.5. Presented with x G S™, we wish to 
read "a few" letters in it to determine whether 5{x,C) is small. 

1. Linearity test. Here E = GF{2) and m = 2" for some integer n. Thus 
S'" is the set of ah functions from Gi^(2)" to GF{2). Let Ci be the set of 
words that correspond to linear functions, namely, the set of / : GF{2)" 
GF{2) such that 3ai,...,a„ e GF{2) s.t. f {zi, Z2, Zn) — J^i'^i^i- The 
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test for linearity involves picking z, u G GF{2)'^ randomly and accepting iff 
f{z) + f{u) = f{z + u). Let 7 be the probability that this test does not accept. 
Using elementary fourier analysis one can show 7 > S(f,Ci)/2 (see also 
earlier weaker results in 

2. Low Degree Test. Here E = GF{p) for a prime p and m = for some n. 
Thus is the set of all functions from GF(p)" to GF{p). Let C2 be the set 
of words that correspond to polynomials of total degree d, namely, the set of 
/ : GF{p)^ GF{p) such that there is a n-variate polynomial g of degree d 
and /(^i, Z2, . . . , , Zn) — g{zi, Z2, ■ ■ ■ , , Zn). We assume dn p (hence degree 
is "low"). Testing for closeness to C2 involves picking random lines. A line has 
the parametric form {(ai + hit, 02 + ^2^, •■•,»« + hnt) : t G GF{py\ for some 

e GF{p). (It is a 1-dimensional affine subspace, 
hence much smaller than GF{py\) Note that if / is described by a degree 
d polynomial, then its restriction to such a line is described by a univariate 
degree d polynomial in the line parameter t. 

• Variant 1: Pick a random line, read its first d + 1 points to construct a 
degree d univariate polynomial, and check if it describes / at a randomly 
chosen point of the line. This test appears in j56j and is similar to another 
test in 

• Variant 2: This test uses the fact that in the PCP setting, it is reasonable 
to ask that the provided certificate should contain additional useful in- 
formation to facilitate the test. We require, together with /, a separate 
table containing a degree d univariate polynomial for the line. We do 
the test above, except after picking the random line we read the relevant 
univariate polynomial from the provided table. This has the crucial ben- 
efit that we do not have to read d -I- 1 separate "pieces" of information 
from the two tables. If 7 is the probability that the test rejects, then 
7 > min{0.1,(5(/,C2)/2} (see % which uses giE]). 

3. Closeness to a small set of codewords. Above, we wanted to check that 
6(f,C) < 0.1, in which case there is a unique word from C in Ball(/, 0.1). 
Proofs of recent PCP Theorems relax this and only require for some e that 
there is a small set of words S C C such that each s G S lies in Ball(/, e). (In 
information theory, such an S is called a list decoding of /.) We mention two 
important such tests. 

For degree d polynomials: The test in Variant 2 works with a stronger guar- 
antee: if (3 is the probability that the test accepts, then there are poly(l/e) 
polynomials whose distance to / is less than 1 — e provided p > poly{nd/ (3e) 
(see 12, and also [M] for an alternative test). 

Long Code test. Here E = GF{2) and m = 2" for some integer n. Thus E™ 
is the set of all functions from GF{2)" to GF{2). Let C3 be the set of words 
that correspond to coordinate functions, namely, 

{/ : GF{2Y' ^ GF{2) : 3i G {1, 2, . . . , n} s.t./(zi, Z2, ■ ■ • , ^„) = z,.} 

(This encodes i € [1, n], i.e., \ogn bits of information, using a string of length 
2", hence the name "Long Code" .) The following test works |37], though we do 
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not elaborate on the exact statement, which is technical: Pick J,w G GF(2)" 
and u G GF{2)" that is a random vector with I's in e fraction of the entries. 
Accept iff f{z + w) — fiz) + f{W + u). (Note the similarity to the hnearity 
test above.) 

3.2. Further applications of PCP techniques 

We list some notable applications of PCP techniques. The PCP Theorem is 
useful in cryptography because many cryptographic primitives involve basic steps 
that prove Yes/No assertions that are in NP(or even P). The PCP Theorem al- 
lows this to be done in a communication-efficient manner. See |42l ISlLirU] for some 
examples. Some stronger forms of the PCP Theorem (specifically, a version in- 
volving encoded inputs) have found uses in giving new definitions for polynomial 
hierarchy and PSPACE 1211 122j. Finally, the properties of polynomials and 
polynomial-based encodings discovered for use in PCP Theorems have influenced 
new decoding algorithms for error-correcting codes , constructions of pseudoran- 
dom graphs called extractors j61[l57) and derandomization techniques in complexity 
theory (e.g. jBOl ). 
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